Developer Testing, Discussions about Software Quality

April: Minneapolis: Build Engineer Bootcamp & CITCON

On this blog when we talk about developer testing we have an ongoing assumption that you're going to use continuous integration in both senses of the phrase: you're going to check-in frequently and you're going to have an automated system like CruiseControl running your tests.

But, perhaps surprisingly, many teams have as much trouble getting the automated system up and running as they do in writing the tests in the first place. The problem is their build. They have a system that is sufficient for assembling their software but that isn't ready for the demands of an agile team, demands like continuous integration, unit tests, functional testing, software metrics, etc.

If you have this problem then you should take a look at the Build Engineer Bootcamp that we have scheduled for Minneapolis in April to coordinate with CITCON, the Continuous Integration and Testing Conference. These two events should pair very nicely: attend the bootcamp to learn the basics, then attend CITCON to delve into the state of the art.

Hope to see you there!

- Posted by Jeffrey Fredrick on March 13, 2009 01:29 PM

Cross-browser testing with SafariWatir and FireWatir

I've written up my account of how I spent my time at Elisabeth Hendrickson's Open Source Test Automation Tool Love In. My goal was to have a single script using Watir to drive both Safari and Firefox. With a bit more work I'd like to have the same script driving Celerity. I like the idea of a continuous integration setup where the Celerity tests run first for fast feedback and then the tests run in the browser to ensure compatibility.

- Posted by Jeffrey Fredrick on February 17, 2009 09:04 AM

CruiseControl 2.8 Released

2.8 is now available for download.

Along with a number of bug fixes and enhancements (release notes) there have been a number of significant below the surface changes that should be invisible out of the box but will be a GoodThing:

Java 5 required to run: this allows us to update the code to more modern Java and to take advantage of the libraries that are part of Java 5 but not Java 1.5. (Of course you can still compile your projects with JDK 1.4.)
Upgraded Jetty: along with the upgrade we are also exposing the Jetty configuration files to allow more extensions to the default behavior.
Updated to Servlet API 2.5 and JSP API 2.1
Specify Log4j config file: there is now a command-line switch (-log4jconfig) to specify the Log4j configuration file. Importantly, this can be either in the .properties format or the .xml format. The .xml format is more verbose (no surprise) but also allows more configuration options. From the Log4j tutorial:

"The properties file does not support some advanced configuration options like Filters, custom ErrorHandlers and a special type of appenders, i.e. AsyncAppender. ErrorHandlers defines how errors in log4j itself are handled, for example badly configured appenders. Filters are more interesting. From the available filters, I think that the level range filter is really missing for property files."

In addition to these change there quite a few little hygiene cleanups that made me happy.

- Posted by Jeffrey Fredrick on November 21, 2008 03:46 PM

Final days to register for CITCON Amsterdam

CITCON Amsterdam is October 3rd & 4th and the final day to register is tomorrow, September 19th. We currently have 124 people registered from across Europe, ready to talk all about Continuous Integration, Testing and related topics. Hope to see you there!

- Posted by Jeffrey Fredrick on September 18, 2008 09:56 AM

Mundane excellence in software, intensive care, and bombing missions

In this blog entry I talk about how simple changes can produce dramatically different results. Developer testing is one of the practices, one of those little things, that add up to excellence:

"… there is no secret; there is only the doing of all those little things, each one done correctly, time and again, until excellence in every detail becomes a firmly ingrained habit, an ordinary part of one’s everyday life."

- Posted by Jeffrey Fredrick on September 15, 2008 10:23 PM

CruiseControl 2.7.2 Released

CruiseControl 2.7.2 is available for download.

Lots of bug fixes, lots of changes to the Dashboard.

New plugins:

Cincom Smalltalk VisualWorks Store Source Control
Git Source Control & Bootstrapper
Http Publisher
Mercurial Source Control & Bootstrapper
SVNLabelIncrementer

See detailed release notes here.

- Posted by Jeffrey Fredrick on April 9, 2008 12:01 AM

Introducing... State Coverage

The other software metric to come out of Agitar, CRAP is simply too much fun. So David Kao (a former Agitar intern) and I decided to take the levity down a notch.* The result?

more »

- Posted by Ken Koster on March 31, 2008 12:41 PM

Register for CITCON Denver

CITCON Denver is just around the corner, April 4th and 5th. This the 3rd annual CITCON North America event, the previous years having been in Chicago (2006) and Dallas (2007). CITCON events have also been held in London (2006), Sydney (2007) and Brussels (2007).

For those who haven't heard about it CITCON is the Continuous Integration and Testing CONference, an open spaces (or "unconference") event that attracts an energetic and enthusiastic crowd of CIT practitioners from around the US, Europe and Asia. If you've never been to an open spaces event you can think of it as the best part of a normal conference -- the hallway conversation -- dialed up to 11!

You can read about past CITCONs on the wiki and then register today. But hurry, there are already 120 people registered and only 30 more registrations will be accepted!

If you can't make it to Denver then you might want to plan ahead by registering now for CITCON Melbourne in June or CITCON Amsterdam in October. Or maybe even all three!

- Posted by Jeffrey Fredrick on March 12, 2008 05:34 PM

Webinar: Making the Business Case for Continuous Integration and Testing

This week on Wednesday March 5th I'll be giving a webinar on Making the Business Case for Continuous Integration and Testing. This webinar is a bit different than previous ones I've given on CIT. I wanted to address this topic because I've talked to a number of people at places like CITCON who believed that CIT was the right thing to be doing but they were inexplicably (to them) unable to convince others that continuous integration and the automated testing that goes with it were something worth investing in. So to try and help people I've put together what I see as the most compelling information and also some advice on how to tailor your case based on specific situation in your company. Following the slide portion of the webinar I'm going to answer questions from the audience. In addition to questions about the slides I'm also interested in hearing from people who've encountered obstacles in adopting CIT and I'll try and provide specific suggestions for getting around the problems. If you're interested in this material you can register for this webinar today.

- Posted by Jeffrey Fredrick on March 3, 2008 11:54 AM

Survey on Business Benefits of Unit Testing

Artur Hildebrandt is a MBA student at the University of Liverpool and currently running a survey on the business benefits and risks of unit testing. His goal is to identify and measure benefits software development organizations can gain by actively practicing unit testing. I think it would be great to have more information available to persuade management on the benefits of unit testing so I'm happy to promote his survey. If you'd like to take part his questionnaire available at online.

- Posted by Jeffrey Fredrick on January 28, 2008 03:58 PM

Continuous Integration at JaSST'08 Tokyo

I'm sitting here in SFO waiting for my flight to Tokyo. I'll be speaking this week at JaSST, the Japanese Symposium on Software Testing. This will be a bit different from my normal talks on CI because I'll be focusing on the benefits of CI for testing groups rather than my normal focus on developers. And there will be translation into Japanese which makes for a very different presentation experience. But I'm very excited about giving this talk because Continuous Integration is less well known in Japan than in Europe, Australia or the US, so it will be fun to be bringing the message to a new audience.

- Posted by Jeffrey Fredrick on January 27, 2008 10:59 AM

AgitarOne available as a downloadable eval!

Several prospects and friends in the software development business have asked us to make a downloadable eval for AgitarOne available on our web site. It is finally here!

more »

- Posted by ashish on January 16, 2008 03:54 PM

Hoorah for XSLT!

It would be handy to have a report of all the unexpected exceptions that get thrown.

I've lost count of the number of times I've heard that. I've even said it myself a few times.

It seemed like it would be pretty easy to do using the dashboard.xml feature of AgitarOne, so I decided to give it a try.

more »

- Posted by Kevin Lawrence on January 14, 2008 04:46 PM

Crap4j 1.1.6 Released

Crap4j 1.1.6 is out. The new version features historical trends of CRAP metrics, and comparison by similarly tagged projects. Exciting!

- Posted by Bob Evans on January 8, 2008 12:06 PM

JUnit Factory is a Jolt Award Finalist!

First we generate over a million free JUnit tests and then we get nominated for a Jolt Award.

What a great end to the first year of JUnit Factory!

- Posted by Kevin Lawrence on January 3, 2008 03:15 PM

Configuring CruiseControl the CruiseControl Way

Configuring CruiseControl the CruiseControl way from ThoughtWorks Studios presents what should be a standard practice. Checking your cruisecontrol configurations into version control and having cruisecontrol bootstrap with them.

Now I could see it getting tricky if you have multiple build machines, but it seems like that would be pretty easy to handle as well by using different config files for different machines, or something similar. That might violate the DRY principle, so it would be good to think a little more about it. How do others deal with this?

- Posted by Bob Evans on November 16, 2007 03:55 PM

Crap4j v1.1.4 Released, Available as an Ant Task now

Like the title says.

This is particularly exciting because now it can be included in continuous integration setups.

One natural next step would be to set a threshold value so that crap4j would throw an error that would fail the build. I would be curious to hear comments on how people would like that to work.

Anyway, find out how to get the latest version at the blog.

- Posted by Bob Evans on October 31, 2007 10:32 PM

Visualizing Complexity and Coverage

At CITCON Europe in Brussels last week one of the sessions I enjoyed was on CRAP4J and other metrics for bad code. (I've put my notes up on the CITCON wiki.) Today Kevin reminded me that Clover has a similar metric for identifying risky code, a tag cloud that uses complexity to size the tag and the coverage level to color it. They have posted a sample using Lucene here. This is a pretty neat looking approach... but honestly? I don't really like it.

more »

- Posted by Jeffrey Fredrick on October 25, 2007 01:12 PM

Crap4j 1.1.3 released

A new version of crap4j has been uploaded to it's new home at http://www.crap4j.org/!

Please check it out and give us your feedback!

- Posted by Bob Evans on October 24, 2007 04:14 PM

CITCON Europe 2007 Starts Today

CITCON Europe 2007 starts today in Brussels, Belgium. We're in a nice central location, very close the Brussels North train station. As an open space event everyone has the opportunity to propose a topic for discussion. The one I'm most interested in is talking about Crap4J and other ideas for metrics to spot bad code.

- Posted by Jeffrey Fredrick on October 18, 2007 11:59 PM

(See the Developer Testing Archives for older posts.)

Highlights

Categories

Recent Entries

Developer Testing:

Sites we RSS